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1 Introduction and preliminaries 

1.1 Introduction 

In many areas of applied mathematics spectral problems for the operator pencil L(A) = 
A — A B arise, where A and B are symmetric or selfadjoint operators acting in some Hilbert 
space 7i. See, for instance, m, (26] , (16] , [29] and references therein. 

One way of dealing with this problem is reducing it to a classical spectral problem for a 
single operator. If B is bounded and boundedly invertible this can be done by introducing the 
operator S = B~ 1 A. It is easy to see that the spectral problems for L and S are equivalent. 
Additionally, one can think of the operator S' as a symmetric or selfadjoint operator in the 
Krein space (%, (B-, •)). Here (H, (B-, •)) denotes the Banach space (%, UlUl 1 / 2 • ||), where 
| B\ 1 / 2 is the positive square root of B , with the structure of the (possibly indefinite) inner 
product on this space given by (B-, •). 

If B is still bounded, but, say, zero is a point of continuous spectrum of the operator 
B, the space (71, (B-,-)) is no longer complete, and S is no longer closed. If we drop the 
assumption that B is bounded, the problems are even more involved, and in the case when 
zero is an eigenvalue of B, S does not exists as an operator. 

There exists an extensive literature on the spectral theory of operator pencils with its 
applications in diverse mathematical and physical domains. Hence, we restrict ourselves to 
mentioning only two classical works (3] and [25]. 

The reduction of the spectral problem for operator pencils to the spectral problem for a 
single operator is a standard procedure used in numerous applications. One of the important 
steps in the reduction is to show that there is no loss of the relevant spectral information. 
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Sometimes, it is sufficient to show that e.g. point spectra coincide, sometimes it is necessary 
to have a complete correspondence of various types of spectra. There is a plethora of such 
results in the literature, but usually for a specific setting and a specific type of spectra. 
Especially important in applications is the point spectrum, and it is treated in several papers, 
usually in different settings. See, for example and m- A recent paper |30] 

contains some results about the correspondence of spectra in the case of bounded operators 
between Banach spaces. As far as we are aware, there are no papers which give a systematic 
treatment of the reduction problem. 

This paper treats the case when the operators A and B are symmetric or selfadjoint 
operators in a Hilbert space and the reduction operator is symmetric or selfadjoint in 
a Krein space /C, its topology generated by the operator B. So the aim is to investigate 
those reductions for which the symmetry of the problem is preserved. We treat various cases 
depending on the properties of the operator B. 

We now give a brief outline of our main results. 

In Sections [2] and [3] we investigate the case when B is bounded and 0 G a c (B). In 
Section [2] we give a construction of a reduction operator in the case when A is a symmetric 
operator with finite and equal deficiency indices. The main result in Section [2] is Theorem [2] 
where we show that there exists a bijective correspondence between all selfadjoint extensions 
of the operator A and all selfadjoint extensions of the reduction operator S. Section E] is 
concerned with the correspondence of the spectra. We show that there is a correspondence 
of spectra and point spectra, but that, in general, there does not exists correspondence for 
residual and continuous parts of spectra. We give necessary and sufficient conditions for the 
correspondence of residual, and hence also continuous, parts of spectra. 

In Section[4]we investigate the case when B is unbounded and 0 G a c (B). A construction 
of the reduction operator follows essentially the one given in |6], but the authors there do not 
investigate the questions we are concerned with, except the case of point spectrum for quasi- 
uniformly positive operators. The main result in this section is Theorem [5] in which we show 
that under the assumptions stated at the beginning of the section, the point spectrum of the 
pencil is contained in the point spectrum of the reduction operator and the spectrum of the 
reduction operator is contained in the spectrum of the pencil. To obtain the correspondence 
of the spectra, additional assumptions are needed, and some are presented at the end of the 
section. 

In Sectional we apply the results from Sections [2] and [4] to obtain a Rayleigh-RitzAike 
upper bound for the eigenvalues of the reduction operator given in terms of the eigenvalues 
of the matrix pair (Ay, By), where Ay, By are orthogonal projections of A, B on a finite 
dimensional subspace. To obtain a variational characterization for the reduction operator, 
we use the results from j5]. 

In Section [6] we investigate the case when 0 is an eigenvalue of B. By the use of the 
theory of linear relations we give constructions of linear relations which act as a substitute 
for reduced operators. We show that similar results as in Sections [3] [4] and [5] also hold in 
this case. Using a result from |28| . we give some examples when a reduced operator can be 
constructed. 
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1.2 Basic definitions and preliminaries 

Let R be a Hilbert space with inner product (•,•) and the corresponding norm ||x|| = 
(x, x) 1 / 2 . By D(T ) we denote the domain, by R(T ) we denote the range of the operator 
T, and by Ker(T) we denote the null-space of T. We say that the operator T is symmetric 
if T C T* and T has a dense domain. The deficiency indices of a symmetric operator T 
are numbers n± = n±(T) defined as n+ = codim R(T — p), n_ = codim R(T — JI). where 
H E C \ R, Qp > 0 is arbitrary. By -j- we denote the direct sum. 

Let L() = A — B be an operator pencil in R, where A and B are densely defined operators 
in R. such that B is A-bounded, i.e. D(B ) D D(A) and \\Bx\\ < C||Ax||. Obviously, 
D(L( A)) = D(A). 

Definition 1. The complex number is in the resolvent set of the pencil L, (denoted by 
E p(L)) if 0 E p(LQ). The spectrum of L is defined as a(L) = C \ p(L), i.e. E <r(L) if 0 £ 
<r(L()). Analogously we define the point a p , residual ay, continuous ay, and approximative 
spectrum a ap . 

We also define the set of the points of regular type of the operator pencil L by 

n(L) = { E C : 3C > 0 such that ||L()x|| > C ||x|| Vx € D(L())}, 

i.e. the set of all E C such that 0 is a point of regular type for the operator L(). 

We say that the vectors xq, x\, ... , x^ € R, xo ^ 0, form a Jordan chain of length k + 1 
for L corresponding to the eigenvalue o € C if 

L(o)xi = Bxi-i, i = 0,1,... ,k, 


where X-\ =0. 

In the next proposition we list some properties of linear operator pencils which are 
essentially known. 

Proposition 1. Let LQ = A — B be an operator pencil in the Hilbert space R, where B is 
A-bounded. 

(i) If L has a compact resolvent for some E p(L), i.e. i/L() _1 is compact, then L has a 
compact resolvent in each point of the resolvent set. 

(ii) If B is bounded, the set of points of regular type of L is an open set. 

(Hi) If B is bounded and A and B are symmetric operators, the deficiency indices of the 
symmetric operators A and A — B coincide, for all E R. 

Proof. The statement (i) easily follows from the generalized resolvent equation. The state¬ 
ment (ii) is evident. For the proof of (iii) see [181 Theorem 9.3]. □ 

The following lemma is known, but we give a proof for reader’s convenience. 

Lemma 1. Let R be a Hilbert space, D a dense subspace ofR, and N a finite dimensional 
subspace ofR. Then D D A r ~ L is dense in N^~. 
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Proof. 0 Let P be the orthogonal projection onto N. Since P is continuous and its range is 
N . the image P(D) is a dense subspace of N. As N is finite dimensional, P(D ) = N. Let Dq 
be a subspace of D such that the restriction Pq : Dq —>■ A of F onto Dq is an isomorphism. 
Then I — P 0 -1 P is continuous on PL, its range is N 2 - and (I — P 0 _1 P)(F>) COfl ASince 
(.I — P ( f 1 P)(D) is dense in N 2 -, so is D n A"- 1 . □ 

Since we will mainly operate in Krein spaces, we introduce some basic notions from the 
Krein space theory. Let 1C be a vector space and let [•,•] be an inner product on AC (not 
necessarily definite). The pair (AC, [•,•]) is said to be a Krein space if there exists a direct 
sum decomposition AC = AC_|_ + AC_ such that (AC±,± [-,-]) are Hilbert spaces and such that 
[AC + , AC_] = {0}. For such a decomposition the corresponding fundamental symmetry J is 
a linear operator defined by J(x+ + x_) = x+ — x_, where x + € AC + and X- £ AC_. Let 
(■, •) = [J-, ■]. Then the space (AC, (•, •)) is a Hilbert space. Its topology is independent of the 
choice of AC+ and AC_. The subspace F is said to be ortho-complemented if F + F 2 - = AC, 
where F* 1 = {x : [x, y\ = 0 Vy € F}. 

The dehnitions of symmetric and selfadjoint operators in a Krein space are analogous to 
those in Hilbert space. A selfadjoint operator A is said to be definitizable if its resolvent 
set is nonempty and there exists a nonzero polynomial p such that \p(A)x, x\ > 0 for all 
x £ D(p(A)). A definitizable operator has a spectral function with finitely many critical 
points in M U {oo}. A critical point z is regular if the spectral function is bounded in a 
neighborhood of z. A critical point is singular if it is not regular. We say that is an 
eigenvalue of positive (negative) type of an operator T if [x, x] > 0 (< 0) for all eigenvectors 
corresponding to . We say that an operator T is quasi-uniformly positive (qup) if there 
exists a subspace M of finite codimension in D(T ) such that 

inf{[Tx,x] : x € M, ||x|| = 1} > 0. 

For more details about Krein spaces see m, 0 and |1]. Basic properties of qup operators 
can be found in im- 

2 A reduction to the single operator problem for A symmetric 
and B bounded 

In this section we assume: 

1. A is a symmetric operator on a Hilbert space PL with finite and equal deficiency indices 
n + (A) = n-(A) =: n, 

2. B be is a bounded selfadjoint operator on PL with 0 G cr c (B), 

3. n(L)nR/0. 

Let F be a spectral function of operator B. We define the operator J by J = F(0,oo) — 
F(—oo,0). Operator J is a bounded selfadjoint operator and J 2 = I holds. We define 
|F| := JB = BJ > 0. Now, we define a space AC which consists of sequences (x n ), x n £ PL 
in the following way 

K, = {(x n ): \B\ l ' 2 x n converges in PL}, 

1 We thank the anonymous referee for this simplified version of the proof. 
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i.e. as a completion with respect to the norm generated by the operator \B\ 1 / 2 . We introduce 
equivalence relation ~ on /C: 


(x n ) ~ {Vn) iff \B\ 1/2 (x n - y n ) 0 as 


n 


oo, 


and dehne 1C as the corresponding quotient space 1C = K/~. On /C we introduce two inner 
products (•, -)fc and as follows: 

{(x n ), ( Vn))K = lim ( \B\ 1 / 2 x n , \B\ 1 / 2 y n ), 

n—>oo 

\{xn), ( 2/n)]/C = lim (J\B\ 1 / 2 x n , \B\ 1 / 2 y n ). 

n—>oo 

It is easy to see that (1C, (•, -)jc) is a Hilbert space, and that (/C, [-, -]/ c ) is a Krein space. 
These spaces have the norm given by ||(x n )||^ = lim n _ ) . (X , || |i?| 1//2 x n ||. If we identify x £ B 
with the sequence (x,x, . ..) in 1C, it is evident that B can be regarded as a subspace in 1C, 
and in that case ||x||^ = |||i?| 1,/2 x|| holds. Also, B is a dense subspace in 1C. Operators in 
B can also be regarded as operators in 1C. Also, J can be extended to a bounded operator 
in 1C, and this extension we also denote with J. It is easy to see that J is a fundamental 
symmetry in 1C. 


Proposition 2. The spaces B and 1C are isomorphic. 

Proof. We dehne 

T : 1C -» B, T(x n ) = lim \B\ 1/2 x r 


(1) 


Then it is easy to see that T is a linear surjection from (1C, (•, •)*;) to ( B , (•, •)) which preserves 
the inner products, i.e. (T(x n ),T(y n )) = ((x n ), (y n ))/C- □ 

In applications, the space 1C can usually be represented as a function space as can be 
seen from the following example. 

Example 1. Let B = L 2 (a,b), and let w € L 00 (a, b ) be a function which is zero on some non¬ 
empty set of measure zero. We dehne the operator B by ( Bf)(x ) = w(x)f(x), f £ L/ 2 (a,b). 
The operator B is obviously bounded and 0 £ cr c (B) holds. 

The Krein space 1C can be identihed with the space 


{/ : VHOI/ e L 2 (a,b)} 

(i.e. a weighted L 2 space with the weight w, see [31} Section 8.4]) with the inner products 
given by 


[f,g\)C= w(x)f(x)g(x)dx, 
J a 

(f,g)lC= [ \w(x)\f(x)g(x)dx 
J a 


The operator T is given by Tf = \/|rc(-)|/, and the fundamental symmetry J is given by 
Jf = (sgn w)f. 

Lemma 2. Let D be a dense subset in B. Then D is also dense in 1C. 
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Proof. Let e > 0, x £ PL be arbitrary. Since R(\B\ l t 2 ) is dense in PL, there exists y £ Pi 
such that || \B\ 1 / 2 y — x|| < and since D is dense in Pi , there exists z £ D such that 
\\y — z\\ < f |||S| 1 /2||- 1 . Now 


\B\ 1 / 2 z- 


< 


\B\ 1/2 (z - y) 


+ 


\B\ l / 2 u- 


y-x 


< £. 


This implies that {|.B| 1//2 x : x £ D} is dense in PL. Now let ( x n ) € 1C, 

e > 0 be arbitrary. Set x = T(x n ) £ PL. Then there is y £ PL such that ||x — |S| 1/,2 y|| < £, 
which implies \\(x n ) — y ||^ < £. □ 

Proposition 3. The operator B~ 1 A is symmetric in (PC, [-, -]yc). 

Proof. Let x,y € D(B~ 1 A) = {x € D(A) : Ax € R(B)}. Then 


[B 1 Ax,y\ K = (J\B\ 1/2 B 1 Ax,\B\ 1/2 y) = (Ax,y) 

= ( x,Ay) = (J\B\ l/2 x,\B\ l/2 B~ l Ay) = [x,B^Ay\ K . 


Let £ II(t)nl be arbitrary. Then R(L()) is closed. From Proposition [T] and |18L Theorem 
9.2] follows codim R(L()) = n. From Lemma [1] follows that R(L())P\R(B) is dense in R(LQ). 
Note that 


R(LQ) n R(B) = {(A - B)x : x £ D(A),Ax £ R(B)} 

= {(A-B)x : x £ D(B~ l A)}. 

Let C > 0 be such that ||L()x|| > C ||x||. Let x € D(A), e > 0 be arbitrary. Set x = L()x. 
Then it exists y £ D(B~ 1 A) such that \\L()y — x|| < Ce. Then 

C£ > || L()y - x\\ = || L()(y - x)|| > C \\x - y\\ , 

hence ||x — y\\ < £. So D(B~ 1 A) is dense in D(A), which implies that D(B~ 1 A) is dense in 
PL. Now Lemma [2] implies that D(B~ 1 A ) is dense in 1C, hence B~ l A is a symmetric operator 
in 1C. □ 

We will need the following three lemmas. 

Lemma 3. Let (x n ) £ D(A*) be a sequence such that 

A*x n -> 0 (2) 

and 

\B\ 1 / 2 x n —> 0. (3) 

Then x n —>• 0. 

Proof. From ([3| it follows that Bx n —> 0. Let A £ II(L)nl be arbitrary. Since (A— \B)* = 
A* — A B, we have 

D(A*) = D(A) + Ker (A* - A B). 
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Let x n = x° + y n , x ° € D(A), y n E Ker(A* — AB). Since ( A* — A B)x n = (A — AB)x°, 
and (A — AB) -1 is a bounded operator, it follows x ^ —>• 0. Hence Bx^ —>• 0 and By n = 
B(x n — x°) —>• 0. Since the operator 

5| K er(A*-AB) : Ker(A* “ AB) ^(Ker(A* _ AB)) 

acts between finite-dimensional spaces and is non-singular, we have y n —>• 0, hence x n —>• 
0. □ 

Lemma 4. We /iaxe 

(i) The set D(|B| _1 / 2 A*) is a core of A*. 

(ii) The set D{\B\~ l / 2 A ) is a core of A. 

(in) For each selfadjoint extension A of A, the set D(|B| _1 / 2 A) is a core of A. 

Proof. Let E n(L) Hi be arbitrary. 

(i) As in the proof of Proposition |3j we have codim R(A—B) = n. Also ( A—B )* = A* — B 
holds. This implies D(A*) = D(A) + Ker(A* — B). 

Let x E D(A*) be arbitrary, and let x = xo + yo, %o € D(A), yo E Ker(A* — B). 
Since Ker(A* — B) C -D(|B| _1 / 2 A*), it is enough to find a sequence (x°) E D(|B| _1 / 2 A) 
such that x^ —>• xq and Ax ° —>• Axo- Lemma |T] implies that B(A — B) n BdBj 1 / 2 ) is 
dense in R(A — B), so for ( A — B)xo there exists a sequence x^ E -D(|B| _1 / 2 A) such that 
(A — B)x° —>• (A — B)xo- Since (A — B)^ 1 is bounded it follows that x° —>• xo which implies 
Bx^ —> Bxo, and consequently, Ax° —>• Axo- 

(ii) Since R(A — AB) n B(B) is dense in R(A — AB), for an arbitrary x € B(A) there 
exists a sequence x n € D(B~~ 1 A) such that (A — \B)x n —>• (A —AB)x, which implies x n —>• x 
and Ax n —>• Ax. 

(iii) Since A is a finite-dimensional extension of A, codimB(A —AB) < n, and R(A — AB) 

is closed. Hence, again from Lemma Q] B(A — AB) n R(\B\ 1 / 2 ) is dense in R(A — AB). Now 
we can proceed as in the proof of the first statement of the Lemma. □ 

Lemma 5. The formulae (|B| -1 / 2 A)* = A*|B| _1 / 2 and (B“ 1 A)* = A*B~ 1 hold. 

Proof. Since | B| - 1 / 2 is bounded, eh Satz 4.19] implies 

(|B| 1/2 (|B|- 1/2 A))* = (|B|- 1 / 2 A)*|B| 1/2 . (4) 

From Lemma |3] follows 

(|B|C2(|B|-1/2 A)) . = (A| d(|b| - ;2a| )* = (^| D( | S |-.,M)) - = A’, (5) 

Now © and (j5]) imply (|B| _1 / 2 A)*|B| 1//2 = A*. The second formula follows analogously. □ 
Let us define the operator S in K. by 

D(S) = {x € D(A) : Ax € R(\B\ 1 / 2 )} = B(|B| _1 / 2 A) 

Sx = T _1 J|B| _1 / 2 Ax, 

where the operator T is defined by ©• 
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Proposition 4. The operator S is closed. 


Proof. Let (x n ) be a sequence in D(S), and let x n —>• x in the norm ||-1|which implies 
x ~ (x n ). Set 

x = T(x n ), (6) 

and let Sx n —>• y in the norm of 1C. It follows 

J\B\-V 2 Ax n ^y:=Ty. (7) 

From (J6]) and (JTJ) follows Bx n —> J\B\ 1 / 2 x and Ax n —»• JIT?! 1 / 2 !/, 
respectively, which implies 

(H - H)x n J| H | 1/2 (y - x), (8) 

for all E C. Especially, (j8j holds for o € n(L), which implies 

x n -> (A- qB)~ 1 J\B\ 1/2 {y - ox) =: x 0 . 

Hence we found xq E Ti such that x n —> xq in the norm ||-|| and Ax n —> J\B\ l ^ 2 y 1 which 
implies xo € D(A) and Ax o = J\B\ l / 2 y. Hence Ax o € R^B] 1 / 2 ), and J\B\~ 1 / 2 Axq = y, i.e. 

Sx o = y and xo E D(S). □ 

Proposition 5. The adjoint of S in K. is given by 

D(S*) = {x E D(A*) : A*x E i?(|H| 1/2 )} = D(\B\~V 2 A*) 

S*x = (y n ), where T(y n ) = J\B\~ l/2 A*x. 

Proof. Let (y n ), (z n ) E /C be such that 

[Sx, (; y n )\K. = [x, (z n )\ K , Vx E D(S). 

This can be written as 

lim(|L>| _1 / 2 Ax, \B\ l / 2 y n ) = lim(J|H| 1//2 x, \B\ 1 ^ 2 z n ) Vx E D(S). (9) 

n n 

Let us denote y = T(y n ), z = T(z n ). Then (0 reads 

(| B\-V 2 Ax, y) = (x, J\B\ l / 2 T) Vx E D(\B\~ 1 / 2 A). 

From Lemma [5] follows y E D(A*\B\~ 1 / 2 ) and 

A*\B\~ 1/2 y = J\B\ 1/2 T. 

Now, since y E i?(|H| 1//2 ) there exists y E TL such that (y n ) ~ y. Hence we can take 
y = |£>| 1 / 2 w, which implies A*y = JlBl^^z. Hence J\B\~ 1 / 2 A*y = z, which implies 
y € D(\B\ V 2 A*) and T(z n ) = J\B\^ 2 A*y. 

On the other hand, take an arbitrary y E -D(|-B| -1 / 2 A). Then one can easily prove 

(| B\~V 2 Ax,\B\ l > 2 y) = (|H|V 2 x, \B\~ l / 2 A*y ), Vx E D(S), 


hence y E D(S*) and T(S*y ) = J\B\~ 1 / 2 A*y. 


□ 


Remark 1. If n = 0, i.e. if A is selfadjoint, then S is also selfadjoint. 

Theorem 1. The operator S is the closure of the operator B~ l A in 1C. 

Proof. Let us denote with S' the closure of the operator B~ 1 A in 1C. 

Let x € D(B~ l A), and set (y n ) = Sx, i.e. lim^-nx, \B\ l ! 2 y n = J\B\~ 1 / 2 Ax € R(\B\ 1 / 2 ). 
It follows that there exists y E Pi such that y r\j (y n ), and |i?| 1//2 y = J\B\ l / 2 Ax, hence 
y = B~ 1 Ax. Hence we have proved B~ 1 A C S, so S' C S. 

Let (y n ) € D((B~ 1 A)*) be arbitrary, where the adjoint is taken with respect to the 
geometry in 1C, and set (z n ) = (B~ 1 A)*x. Then 

[H _1 Ax, (; y n )] K = [x, ( z n )\ic 

holds for all x € D(B~ 1 A). This implies 


(1 B\~ 

1/2 Ax, y) = (x, J\B\ l / 2 z), Vx € D(B~ l A), 

(10) 

where y = T(y n ), z = T(z n 

). The relation (11011 can be written as 



(B~ 1 Ax,J\B\ 1 l 2 y) = (x,J\B\ 1 l 2 lf). 

(11) 


From Lemma |5]we know that (B l A)* = A*B 1 . hence (fTTTl implies 

A*\B\~ 1/2 y = J\B\ l/2 z. 

This implies y € T?(|i?| 1//2 ), hence (y n ) ~ y € Ti and A*y = J\B\ 1 / 2 z, which implies 
y € D(\B\~ 1 / 2 A*). We have proved that (B~ 1 A)* C S*, hence S C (B~ 1 A)** = S'. □ 

Let 5 be a closed symmetric operator in a Hilbert space with equal deficiency indices. 
It is well-known that the set D(S) provided with the inner product = (^>2/) + 

(S*x, S*y ) is a Hilbert space. 

Definition 2 f |27| ■ |13|.|19j). Let S be a symmetric operator in a Hilbert space %. A 
collection {T2,Ti,T 2 }, in which Cl is a Hilbert space, and ri,r 2 : D(C*) —>• Cl are bounded 
operators, is called a space of boundary values (SBV) for S*, if 

1. (S*x, y) — (x, S*y) = {Tix,r 2 y)n — (C 2 X,Tiy)Q Mx,y <E D(S*), 

2. the mapping T : x i —> -{Tix, T 2 x} from D(S*) to Cl x Cl is surjective. 

In the case of a symmetric operator S in a Krein space we have 

Definition 3 ([2DJ). Let S' be a symmetric operator in a Krein space (1C, [•, •]). A collection 
{H, J, Ti, T 2 }, in which Cl is a Hilbert space, J is an involution in Cl, and Ti, T 2 : D(S*) —>• H 
are bounded operators, is called a space of boundary values for S*, if 

1. [S*x, y] - [x, S*y] = (r^, JT 2 y) n - (T 2 x, Jr iy ) 0 Vx, y € D(S*), 

2. the mapping f:a:4 {Fix, T 2 x} from D(S*) to Cl x Cl is surjective. 
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In the case of Hilbert space operators, it can be shown that a SBV exists for any sym¬ 
metric operator with equal deficiency indices (see |13j . [19]). In the case of Krein space 
operators, it can be shown that a SBV exists for an operator T if n(T) 7 ^ 0 holds, and if the 
deficiency indices of T are equal (see |20|). For a fixed SBV for the operator T, there exists 
a bijective correspondence between the collection of the closed extensions T and the set of 
closed relations 6 in H: 

x e d(t) = d(Tq) <*=*. {r 2 ®, Tix} eecfixfi. (12) 

An extension T is selfadjoint if and only if 0 is a selfadjoint relation, or equivalently, if there 
exists a selfadjoint operator G on Fl given by the relation 

(cosG)r 2 x — (sinG)ri.T = 0 <(=^ {r 2 x,Tix} G 6 . 

For more details on the theory of SBV see |I9], [20 . [13] and |2?J- 

Let {S2, Ti, r 2 } be a fixed SBV for the operator A*. Set F, = r? — 1 / 2 ^ 4 *), i = 1,2. 

We will show that a collection {V2, V, Ti, T 2 } is a SBV for the operator S*. Hence, we need 
to show that the statements 1. and 2. from Definition [3] hold. We start with 

[S*x, y] K ~ [x , S*y] K = (| B\~V 2 A*x,\B\ l / 2 y) - {J\B\ l ' 2 x, J\B\~ 1 / 2 A*y ) 

= (A*x,y) - (x,A*y) = {Tix,T 2 y)n - (T 2 x,Fiy)n 
= (fix,f 2 y) n - (r 2 x,fi y)n. 


To see that F, is bounded for i = 1,2, it is sufficient to prove that there exists C > 0 such 
that 


and since 


\ D (A-) < C1M|d ( s*) VxeD(S*), 


B\ 1 / 2 I B\~ 1/2 A* 


||A*x|| < \B\ l / 2 \B\ l t 2 A*x < \ar~ \&\ ~'~A'x 
it is enough to see that there exists C > 0 such that 

INI < C ( \B\ l / 2 x +||A*s||) Vx € D(|H| _1 / 2 y4*). 


(13) 


But this is implied by Lemma E] Indeed, suppose that (1131) does not hold. Then there is a 
sequence (x n ) € D(\B\~ 1 / 2 A*), ||x n || = 1, such that 


> n 


Hi 1 / 2 


Xr 




which implies A*x n —>• 0 and \B\ 1 / 2 x n —>• 0, hence x n —>• 0, which is a contradiction with the 
statement of Lemma [3] 

The density of D(S*) in D(A*) follows directly from Lemma [4] Our next aim is to show 
that r = {ri,r 2 } is surjective. Let f,g G H be arbitrary. Then there exists x G D(A*) such 
that Tix = /, T 2 x = g. Since D(S*) is dense in D(A*), there exists a sequence (. x n ) G D(S*) 
such that x n —>• x in the norm of D(A*), and since Tj, * = 1,2 are bounded, it follows that 
r iX n = TiX n —>• TjX, i = 1,2, so R(T) is dense in H x H. Now, since the space 11 x is 
ffilite-dimensional, it follows R(T) = SlxSl. 
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Hence, we can parameterize the selfadjoint extensions of the operator S by the use of 
a SBV for the operator A*. More precisely, let A = Ag be an extension of the operator A 
generated by the closed relation 6 on O. We define an extension S' of S' in the following way: 

D{S) = D{\B\~ l / 2 A) and Sx = S*x. 

This definition implies 

D(S) = {x € D(A) : lx = M*x € tf(|H| 1/2 )} 

= {x € D(A*) : A*x € R(\B \ 1/2 ) : {T 2 x, Tix} € fl} 

= {x € D(S*) : {f 2 x,f lX } € £(S 0 )} . 

On the other hand, if S = Sg is a closed extension of the operator S generated by a closed 
relation 0 on 0, then it generates a closed extension Ag of the operator A. Thus, we have 
shown 

Theorem 2. There exists a bijective correspondence between all closed extensions of the 
operator A and all closed extensions of the operator S. Specifically, there exists a bijective 
correspondence between all selfadjoint extensions of the operator A and all selfadjoint exten¬ 
sions of the operator S. Moreover, if A is a selfadjoint extension of A, then the operator S 
defined by D(S ) = D{\B\~ l / 2 A), Sx = {y n ), T(y n ) = J\B\~ l / 2 A, is a selfadjoint extension 
ofS. 

Rem.ark 2. In the case when a selfadjoint extension of A is given by the relation (cos G)r 2 x — 
(sin G)T\x = 0, where G is a selfadjoint operator in H, the relation (cos G)T 2 x— (sin G)Tix = 
0 defines a selfadjoint extension of S. 

Example 2. Let Ti = L 2 (0,1), Af = D{A) = C 2 (0,1), (Bf)(x) = x/(x). The 

operators A and B are obviously symmetric, B is bounded and 0 € a c (B). One can easily 
see that n(L) / 0. A SBV for A* can be chosen as follows: 

n = c 2 , r, = {-/(o),/(i)}, r 2 = {/(o),/'(i)}. 

We have D(A*) = Wf (0,1), where Wf (0,1) is the usual Sobolev space, and 

D(\B\-V 2 A*) = j/ G Wf (01,) : -L0(x) € L 2 ( 0 , 1) j . 

The collection {C 2 , /, Ti|_ d(|^|— 1 / 2 ^ 4 *), T 2 — 1 / 2 ^ 4 *)} is a SBV for S*, hence all selfadjoint 
extensions of S are parameterized by the selfadjoint relations in C 2 by the formula ( 1121 ) . 


3 The correspondence of the spectra in the case of bounded 

B 

In this section we have the same assumptions as in the previous section. Our aim in this 
section is to show the connection between the spectra of L and S. 

Let 0 be a selfadjoint relation in 0 X P such that p(L) 7^ 0 holds, where L() = Ag — B = 
A — A B, and let S = Sg be a selfadjoint extension generated by the relation 6. 
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Proposition 6. The operator S is not bounded. 

Proof. Let us assume that S is bounded. This implies D(S ) = 1C, hence D^B^ 1 / 2 A) = 1C, 
which implies 1C = Pi. This implies that for each sequence ( x n ) G PL such that \B\ 1 / 2 x n —>• 
y, there exists x G PL such that x ~ (x n ), i.e. | B\ l / 2 x = y. Hence R(\B\ 1 / 2 ) = PL, a 
contradiction with the assumption 0 G a c (B). □ 

For a symmetric operator S in a Hilbert or Krein space, by ir(S) we denote the number of 
negative squares of Hermitian sesquilinear form ( Sx,y ), x,y € D(s), where (•, •) is the inner 
product on the Hilbert or Krein space. That is, n(S) is the supremum of the dimensions of 
all subspaces L such that [Sx, x) < 0 for all 0 A x € C. 

Theorem 3. For each selfadjoint extension A of A, and S the corresponding extension of 
S, the formula n[S) = it {A) holds. 

Proof. The inequality 7 r(S) < ir[A) is evident. 

We will first prove the theorem in the case tt[S) = 0. In this case we have [Sx, x]/c > 0 
for all x G D(S), or, equivalently [Ax,x) > 0 for all x G D(\B\~ 1 ^ 2 A). Let us assume that 
tt(A) > 0. Then there exists some x G D[A) such that (Ax,x) < 0. Lemma 0] implies 
that there is a sequence (x n ) G -D(|-B| — 1 / 2 T) such that x n —>• x and Ax n —>• Ax, hence 
(Ax n ,x n ) —>• (Ax,x), so for n large enough we have ( Ax n ,x n ) < 0 ; a contradiction. 

Now we treat the case tt(S) > 0. Let integer n and elements x\,...,x n G D(A) be 
arbitrarily chosen. Let A be the matrix ((Ax n , x n ))fj =1 , and let k be the number of neg¬ 
ative eigenvalues of A. Then, by Lemma [4] (iii), for each e > 0 there exist x\,... ,x' n G 
D(\B\~ 1 / 2 A) such that ||x* — x'|| < e and || A(xi — x'f)\\ < e. Since 

| (Axuxj) - {Ax[, x'j)\ = | {A(xi - x'fj^j) + (Ax^xj - ®')| < 

< e(||xj|| + \\Axi\\) + e 2 , 

for a sufficiently small e > 0 the matrix A! = {fAx\, x' ))" J=1 will have at least k negative 
eigenvalues. Hence, the number of negative squares of the Hermitian form ( Ax,y ), x,y G 
D{A) is smaller or equals the number of negative squares of the Hermitian form [Sx,y]/c, 
x,y G D(S), which finishes the proof. □ 

Theorem 4. The spectra of S and L coincide. Moreover, their point spectra coincide, as 

well as their eigenspaces and Jordan chains. 

Proof. First we will show a p (L ) = a p (S). Let G cr p (L) be arbitrary. Then there exists 
x G D(A), x/0 such that 

Ax = Bx. (14) 

It is evident that x € D(S). Multiplying ffl4l) by J|H | -1 / 2 we obtain 
J\B\~ 1 / 2 Ax = (HI 1 / 2 , which implies T(S — )x = 0, hence (S — )x = 0. 

On the other hand, for G cr p (S) there exists x G D(S), x A 0, such that 

J\B\- 1/2 Ax = \B\ l / 2 x. (15) 

Multiplying (H5l) by J\B\ 1 / 2 we obtain Ax = Bx, i.e. G a p (L). Hence, cr p (L) = cr p (S). 
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( 16 ) 


The relation (S — )x\ = xq implies 

J\B\~ l /' 2 A Xl - IBI 1 / 2 ®! = \B\ l / 2 x 0 . 

Multiplying (1161) by J\B\ 1 ^ 2 we obtain {A — B)x i = Bx o, hence a Jordan chain of S corre¬ 
sponding to an eigenvalue is also a Jordan chain of L for the same eigenvalue . The other 
direction can be seen analogously. 

Let £ P(J?)- This implies T(R(S — )) =R, i.e. 

R(J\B\~ 1 / 2 A — |H| 1//2 ) = R. This implies 

R(A-B) D R(\B\ l > 2 ) D R(B). (17) 

Now, let p £ p{L) be arbitrary. From (1171) follows 

(A — pB)x = {A — B)x + ( — p)Bx £ R{A — B), 

hence R(A — pB) C R{A — B), which implies £ p(L). 

On the other hand, let € cr(S') \ a p (S). Then there exists (y n ) £ K, such that 

(y n )$R(S~). (18) 

We denote y = T{y n ). Then, the relation (1181) can be written as 

y i R(J\B\~ l / 2 A- |5| 1/2 ) 


which implies 

J\B\~ l / 2 y i R((A - B)\ Dm _ 1/2A) ). (19) 

Let us assume that J\B\ l / 2 y £ R(A — B). Then there exists x £ D(A) such that Ax — Bx = 
J\B\ 1 / 2 y, Jience Ax £ R(\B\ 1 / 2 ), a contradiction with (fl9l) . Hence, J\B\ 1 / 2 y ^ R(A — B) 
and £ <j(L). □ 

Proposition 7. We have 

(i) a c (S) C a C (L), 

(ii) d a p (S) c & ap (L). 

Proof, (i) Let € cr c (S). Then R(S — ) is dense in 1C. For each (y n ) £ /C, and for each £ > 0 
there exists x £ D(S ) such that 


(S-)x 


(yn) 


< £. 
K 


We choose y £ R(B) arbitrarily and set y = J\B\ 1 / 2 y, y' = \B\ 1 ! 2 y. For y' £ R C 1C 
there exists x £ D(S ) such that IKS' — )x — y'\\ K <e\\\B\^ 2 \\-\ or, equivalently, 


J\B\~ X ! 2 Ax — \\B\ l / 2 x — y 


< £ 


\B\ 1 / 2 


-l 


Now, 


(A — B)x — y = J|H| 1 / 2 (J|H|- 1 / 2 Hx- \B\ l / 2 x-y) 


< £. 
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Hence, R(A — B ) is dense in R(B), which implies that R(A — B ) is dense in R. Since 
^ ( 7 p (L ), € <J C (L) follows. 

(ii) Let € a ap (S). Then there exists a sequence (x n ) £ D(S), ||^ = 1 such that 

IKS' — )x n || —> 0 , or equivalently, 

(J|H|- 1 / 2 1 -|H| 1 / 2 )x n -> 0 ( 20 ) 


and 



( 21 ) 


Multiplying relation fl20l) by J\B\ 1 / 2 we obtain (A — B)x n —t 0. From fl2TT) we see that 
||x n || > |||i?| 1 / 2 || . Set x n = x n /||x n ||. Then ||x n || = 1 and (A — B)x n —>• 0, hence 

€ (T a p(L). □ 


Corollary 1. The relation oy(L) C a r (S ) holds. 


In general, Proposition 0] (and hence also Corollary [T]) cannot be improved, as will be 
shown in Example [3] 

First we give sufficient and necessary conditions for the complete correspondence of the 
spectra of L and S. 

Lemma 6. The subspace R(A — B ) is dense in TL if and only ifTLn R(S — )■*■ = {0}. 

Proof. ==^\ First we show that the subspace R(A — B ) is dense in R if and only if R((A — 
b )\ D (\b\-i/2A)) is dense in n - 

The case = 0 follows from Lemma 01 so we assume 7 ^ 0. First we assume that R{A — B) 
is dense in R. Let xo £ R, £ > 0 be arbitrary. Then there exists x £ D(A) such that 
||(M — B)x — Xo|| < e/2. Lemma 0] implies that there exists x' £ -D(|S| -1 / 2 ^) such that 
||x — x'|| < 4 |||| B |i and || A(x — x')|| < e/4. Now, 


(A - B)x' 


x 0 


< 


(. A-B){x' 


x) + {A — B)x — xq 


< e. 


The other direction is obvious. 

We showed that R((A — -S) I z?(|S|—i/ 2 ^) ) i s dense in R , hence the relation 

(x, (A — \B)y) = 0, Vy£D(\B\- 1 /' 2 A) 


implies x = 0. Now, from 

(x, (1 - A B)y) = (\B\ 1,2 x, \B\-W{A - A B)y) = [x, (S - A )y] K 
our claim immediately follows. 

The other direction follows from the similar reasoning. □ 

Lemma 7. The subspace R(S — ) is dense in 1C if and only if 
R{\B\^ 1 ^ 2 {A — B)) is dense in R. 
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Proof. =>: Let us assume that R(\B\ l / 2 (A — B)) is not dense in PL. Then there exists 
x £ PL, x / 0 such that 

(x, \B\~ l / 2 {A - B)y) = 0 \/y £ D(\B\~ l / 2 A). 

Set (x n ) = T~ 1 Jx. Then ((x n ), (S' — )y)/c = 0 for a ll V € B(S), which implies the statement. 

Let us assume that R(S — ) is not dense in /C. Then it exists (y n ) £ /C, (y n ) ^ 0, such 
that ((y n ): (S — )x)tc = 0 for all x £ D(S). Set y = T(y n ). Then 

(y, (J|S|- 1/2 1 - \B\ l / 2 )x ) = 0 Vx € D{\B\~V 2 A), 

hence 

=0 Vx € D(\B\~ l / 2 A), 

and the statement follows. □ 


Lemmata [ 6 ] and [7] imply the following result. 

Proposition 8. (i) If £ a r (S), then £ <x c (L) if and only if 

nn r(s-) ± = {o}. 

(ii) If £ a c (L), then £ a r (S) if and only if 1 / 2 (v4 — B)) is not dense in PL. 

Example 3. Let PL = £ 2 ( 0 , 1). We define the operators A and B in PL by: 

(a/o(x)= 

j 0 

(■ Bf)(x ) = (2x - 1 )/(x), 


where 


G(x,0 


X , X < 

t ,*>£ 


Note that A is the inverse of the Dirichlet Laplacian. The operator B is bounded and 
0 £ a c (B), 0 £ cr c (A) holds. Note that this implies 0 £ cr c (L). One can easily prove that 
n(L) n R / 0 holds. 

We will show that R{\B\~ 1 / 2 A) is not dense in PL. Indeed, constant functions are not 
contained in R(\B\~ 1 / 2 A). This follows from the fact that the function x 1 — > \J\2x — 1| is 
not contained in R{A). Now Proposition [8] (ii) implies 0 £ a r {S). 


Remark 3. If tt(B) < 00 or A > 0, then a r (L) = a r (S ) = 0, and consequently, cr c (L) = <J C (S). 
If S is definitizable, then a r (S) = 0, hence oy(L) = 0 and cr c (S) = cr c (L). 


Corollary 2. Let us suppose that ir(A) < 00 . Then each self adjoint extension S of S is 
definitizable. 


Proof. Since deficiency indices of A are finite, vt ( j 4 ) < 00 holds for each selfadjoint extension 
A. and Theorem [3] implies 7 r(S) < 00. Now, let 0 £ II(L) Pi R be arbitrary. It is well-known 
that there exists a selfadjoint extension Aq such that 0 £ p(A$ — qB) (see [2T], [13]), which 
implies 0 £ p{Sq), where Sq is the corresponding operator. Also n(So) = tt(Aq) < 00 holds. 
Now, from [101 Proposition 1.1] follows that each selfadjoint extension of S has a nonvoid 
resolvent set, hence, see [23], each selfadjoint extension of S is definitizable. □ 
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Remark 4. If A is qup, the proof of Lemma 3.1 from | 6 ] implies 0 E p(L) for ^ 0, | 
sufficiently small. Hence E p(S) for such . From this fact and from n(S) = tt(A) < oo, it 
follows that S is also qup. 

Corollary 3. If L has a compact resolvent in one point (and then, from Proposition [7J in 
all points), and if 

Ax = Bx, x 0 =>■ (Bx, x ) / 0, ( 22 ) 

then S has a discrete spectrum. 

Proof. See @]. □ 

Rem.ark 5. The relation (1221) is equivalent with the fact that S does not have isotropic 
eigenvectors, i.e. if \x, x\ 0 for all eigenvectors x of S. 

4 B unbounded 

In this section we will treat a more general case when B is possibly unbounded. In this 
section we assume: 

1. A and B are selfadjoint and D(A) C D(B), 

2. 0 is not an eigenvalue of B, and 

3. p(L) flR^O. 

Using a spectral shift (i.e. substituting A by A — XB), if necessarily, we can assume 0 € p(A). 
The construction which we use here is essentially given in j 6 |. 

Similarly as in Section [2] we introduce a space 1C as the completion of (D(|£>| 1//2 ), [-, •]), 
where [■,•] = (JI.BI 1 / 2 -, •), by the use of Cauchy sequences. As in Section [21, we set J = 
E(0,oo) — E(—oo,0), where E is the spectral function of B. By T we denote the operator 
T : 1C —» Pi, T{x n ) = lim n \B\ l / 2 x n . It is easy to see that T is an isomorphism between 
spaces PL and 1C. As was shown in [6], A~ 1 B extends to a bounded symmetric operator R in 
1C, and R is injective. Hence, the operator S = R is selfadjoint and boundedly invertible 
in 1C. Moreover, we can see that S is an extension of B~ l A which is densely defined in 1C. 
Indeed, since R is injective, it holds 

(lFU4) -1 = (H-M)- 1 = R, 

so B~ 1 A = S, where the closure is taken in 1C. The domain of S is given by 

D(S) = {(s n ) € K : J\B\-^Ax n is convergent in PL}. 

For (x n ) E 1C we can always assume x n E D(B). Then, if (y n ) = (S — )(x n ) we have 
T(y n ) = lim n J\B\~ 1 / 2 (A - B)x n . 

If A is a qup, we can, instead of the assumption 3., make the assumption p(L) 0, 
since in [BJ Lemma 3.1] it was proved that in this case there exists o such that L(q) is 
boundedly invertible. Also, in this case, S is a qup. 
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Theorem 5. We have 


cr p (L) C <Jp(S) 


(23) 


and 

a(S) C cr(L). (24) 

Proof. The relation fl23l) is obvious. 

Let G p(L). Then it is easy to see that l-Bl 1 / 2 ^ — B\~ 1 / 2 and li?! 1 / 2 ^ — B\~ 1 / 2 are 
everywhere dehned bounded operators. From 

(\A - B\~ 1/2 \B\ 1/2 y = \B\ 1/2 \A - B\~ 1/2 

follows 

(|-4 - BY 1/2 \B\ 1/2 )** = ( \B\ 1 / 2 \A - B\~ 1/2 )*, 

hence 

\A - Hr 1/2 |H| 1/2 C (|H| 1/2 |A. - BY 1 / 2 )*. (25) 

Since the right hand side in fl25l) is a bounded operator, it follows that also the left hand 
side in (12511 is a bounded operator (but, in general, not everywhere dehned!). Hence, the 
operator 

|B| 1/2 {A - B)~ 1 \B\ 1 / 2 = IBI 1 / 2 ^ - B\~ 1/2 sgn(A -B)\A- BY 1/2 \B\V 2 
is bounded. 

The subspace D^Bf 1 / 2 ) n R^Bl 1 / 2 ) is dense. Indeed, we have 

D(IB] 1 / 2 ) n Rm 1 / 2 ) = {x = \B\ l ' 2 y : y G D(B)}, 

and the right hand side is dense since D(B ) is a core of {Bl 1 / 2 . Now, let x G PL be arbitrary. 
Then there exists a sequence (x n ) G D^B^/ 2 ) n R(\B^-/ 2 ) such that x n —>• Jx. From the 
considerations given above it follows that 
\B\ 1 / 2 (A — B)~ 1 \B\ 1 / 2 x n converges to some y gE 

Set z n = (A — B)~ 1 \B\ 1 / 2 x n . Then \B\ 1 / 2 z n —>■ y, hence (z n ) G 1C and x n = |F>| -1 / 2 (4.— 
B)z n . Now \BY 1/2 {A — B)z n —> Jx , which implies J\BY 1 / 2 {A — B)z n —> x , so x G 
T{R(S -)). This implies G p{S). □ 

The other inclusion a(L) C cr(S) in general does not hold, since for large the operator 
L() need not be closed. 

If we assume 0 G p(B), then 1C = D^Bl 1 / 2 ) and cr p (S) C <r p (L), hence cr p (S) = a p {L). 
Also, it can be seen that cr ap (L) C o' ap (S). Hence, if L has only approximative spectrum 
then cr(L) = cr(S). 

Another kind of construction is made in |24| . There it is assumed that 0 G p(B) and 
D(A) = D^B] 1 / 2 ). Then L() is a holomorphic function in Kato sense from 1C to R. For the 
readers convenience, we give a definition. 

Definition 4. The function T() : X —>• y is holomorphic in Kato sence if there exist a 
bounded holomorphic functions U() : Z —>• X and K() : Z —>• y such that T()U() = V() 
and R(UQ) = D(T()) for all G C, where Z is some Banach space. An operator function 
T() is bounded holomorphic if it takes values in the set of bounded operators and if T()x is 
a holomorphic function in for all x. 
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In our case, we can choose U() = B -1 , V() = AB~ l — , and Z = 'H. 

We set S = B~ 1 A, and by the use of the similar methods as in the proofs of Theorem 
[4] and Proposition [TJ it can be seen that a(L) = er(S'), a ap {L) = cr ap (S), cr c (L) = a c {S) and 
tT p (L ) = (jp(S), hence we have the complete equivalence of the spectra of L and S. 

5 Variational characterization 

In this section we apply the constructions of reduced operators from Sections [2] and [4] to 
obtain a Rayleigh-Ritz-like upper bound for the eigenvalues of the reduction operator given 
in terms of the eigenvalues of the matrix pair {Ay,By), where Ay, By are orthogonal 
projections of A, B on a finite dimensional subspace. Hence the bound is given in terms of 
the original data (A,B), which can be important in applications. 

To obtain a variational characterization for the reduction operator, we need a result from 

u 

Let Q be a qup operator such that all finite critical points are regular, and such that 
there are no critical points embedded in the positive continuous spectrum. For such a Q, in 
|5j a variational characterization of eigenvalues is obtained. More precisely, we define 

f = sup {inf : x € M n C ± n £>(Q) j : M € Mj 1, (26) 

where C±- = {x : [x,x] ^ 0}, and A4j denotes the set of all subspaces of codimension j — 1. 
Then, if by d ^ we denote the positive (negative) spectral shift of Q (for more details see 
ED, a number ± +d ± is either an eigenvalue of Q, or a point on the boundary of the essential 
spectrum of Q, for all j € N. 

If the operator S from Section [2J or the operator S from Section [4] satisfy conditions 
stated above, we can variationally characterize the eigenvalues of S or S, respectively. If B 
is bounded, then we have the following variational characterization 

t(S)=f(L) 

= sup |inf | ^ ^ : x € M n C± n D{\B\~ l / 2 A) j : M E m\ . (27) 

In the case when B is unbounded, we have 

[^(Xn), (Xn)]/C _ j. {AXm X n ) 

[(x„), (x n )]/c n {Bx n ,x n y 

where ( x n ) € D{S ). Since D{\B\~ 1 ^ 2 A) is a core of the operator S, from 112 Theorem 4.5.3.] 
follows that in this case we have 

t(S)=f(L) 

= sup |inf 1 X J : x € M FC^ (1 D(\B\~ 1/2 A)^ : M € Mj j . (28) 

Now, suppose that the operator S from Section [2] satisfies conditions stated above. Let V C 
D{\B\~ l t 2 A) be a finite dimensional ortho-complemented subspace in /C. Let dimR = N. 
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Denote by P a orthogonal projector in /C onto V. Set Ay = PAP, By = PBP. Let _L 
and [_L] denote the orthogonal complement in P and 1C, respectively. The set of ortho- 
complemented subspaces of 1C we denote by O. Let F be a subspace of V, with dimF 1 = k, 
and let F = spanjxi,..., Xk }• Set F' = span-fL^xi,..., B^Xk}- Then dim F' = k and 

x[-L]F' -<=>- x-LF, Vx € V. 

On the other hand, if x[_L].F, then we define F" = span{£>yxi,..., ByXk} and then we have 

x[±\F X-LF", Vx € V. 


For m < N, set 


Bm — 


sup . 

FCV ( ByX,X) 

dim F=m—1 x±F 
XGC+ 


inf 


{AyX, X) 


Then, from the considerations given above, we have 

. f (Ax,x) 

Bm = sup inf —- r 

F<zV XGV [Bx ,X) 

dimF=m— 1 

xGC + 


■ , (Ax,x) 

sup mt — - T 

yi,—,ym- iGV xev (Bx,x) 

spanfei.y m _i}60 zGspanjyi.. ,y m - 1 }^ 

x£C+ 

. , {Ax, x) 

sup inf — -r 

yu -,ym-iGK rn xeV n 

span{j/i xGspa,n{Pyi,...,Py 7n —i}i- L > 

x€C+ 

■ , (lx,x) 

sup mf — -- 

yi,-,Vm-ielC r *ev (Bx,x) 

span{i/i,...,y m _i}GO a: Gspan{yi,...,y m _i}l- L ) 

x€C+ 


> 


(29) 


> sup inf 

yi ,..., yrn -iGK xeD(\B\-V 2 A) 

span{yi,...,y m _i}GCl x£span{j/ii} W 

x£C+ 


{Ax, x) 
{Bx, x) 


= sup 

MeM-i 


inf 

veD(\B\- l FA) {Bx,x) 
XGMHC+ 


The analogous relation can be obtained for the eigenvalues of the negative type. From the 
finite dimensional principle given in |7j, fi m is an eigenvalue of the matrix pair {Ay, By). 
Hence we obtained a Rayleigh-Ritz-like upper bound for eigenvalues of S: 


f{Ay,By)>f{S). 

The analogous Rayleigh-Ritz upper bound holds also for the operator S from Section [4] due 
to the formula fl28l) . 
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6 Zero is an eigenvalue of B 


If 0 is an eigenvalue of B, then in general a reduced operator cannot be constructed. But we 
can construct an reduced linear relation, a generalization of the notion of linear operator, 
and we can recover the results from previous sections. Under some additional assumptions 
we can also construct a proper reduction operator. 

In this section we assume: 

1. A and B are selfadjoint operators, 

2. B is ^-bounded, 0 G a p (B), 

3. p(A, B ) nM / 0. 

The last relation implies that we can assume 0 G p{A). 

Then B~ 1 A does not exist as a linear operator, but it can be introduced in terms of 
the linear relations, i.e. as a subspace of R x R (for the basic definitions see [28], [14]). We 
define 

S 0 = B~ X A = {{x,y} G D(A) x D(B) : Ax = By}. 

By S we denote the closure of So as a subspace in R xR. 

Let 1C denote the factor space D(B) /xer B, and let 1C denote the completion of (/C, (£»•, •)). 
Evidently, 1C is a Krein space, and it consists of all Cauchy sequences in D{\B\ 1 ^ 2 ), where we 
identify such two sequences (. x n ) and (y n ) if \B\ 1//2 (x n — y n ) —>• 0. This equivalence relation 
we denote by the ~ symbol. Let [x] denote a class in 1C represented by x. 

Let R(, S ) be a resolvent of S. Then the family of linear operators Rf, S ) in TL induces a 
family R(,S) of linear operators in the space 1C, since R(,S) Ker B C Ker B (see [24]). We 
define the linear relation S in 1C by S = R(0, S)^ 1 . 

We say that is a discrete eigenvalue of the operator T if is an isolated eigenvalue of 
finite multiplicity of the operator T. 

First we treat the case when B is a bounded operator and zero is a discrete eigenvalue. 
Then it is easy to see that we can substitute Cauchy sequences with elements of R, and that 
x ~ y if and only if x — y G Ker B. The inner product on K is given by [•, •] = ( B- , •). It is 
easy to see that 

S = {{[x], [y]} : x — A~ l By G Ker 5}, 
and that S' is a selfadjoint relation. 

Theorem 6. The spectra of S and L coincide. Moreover, their point spectra coincide. 
Proof. The relation er P (L) C cr p (S ) is obvious. On the other hand, let 

G cr p (S). Then there exists iGU such that x — A~ 1 Bx G Ker B, hence Bx— BA~ 1 Bx = 0. 
There exists y ^ 0 such that Bx = Ay, hence 

0 = (/ - BA~ l )Bx = (I - BA~ l )Ay = Ay - By, 

which implies G a p (L). It is easy to see that 

(S - B 1 = {A- B)~ l B V G C \ a p (L) =C\ a p {S). 

From this relation it is easy to obtain p(L) = p(S). □ 
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If B is bounded, but zero is not a discrete eigenvalue of B, it can be seen that 
S = {{[x], [(y„)]} : x - A -1 J\B\ 1 / 2 lim \B\ l/2 y n € KerS}, 

n 

where J is dehned as in section [21 is a selfadjoint relation. Using similar techniques as in 
Section [2] it can be shown that <7(S') = cr{L) and cr p (S) = u p {L). 

This kind of procedure can be implemented also in the case when B is not bounded, 
using the techniques from Section [4] For instance, if all assumptions given in the beginning 
of the Section [4] hold, except from 0 ^ a p (B), we can define S by 

s = {{[(x n )], [(s/n)]} : (x n ) ~ {A -1 By n )}. 

Reasoning analogously as in the proof of Theorem [5] it can be shown that cr p (L) C <7 p (5) 
and a(S) C <r(L). 

When is it possible to reduce the relation S to an operator, without the loss of the 
information about spectra? To answer this question we introduce the notion of the multi¬ 
valued part Too = {{O,^} € T} of the relation T (see [28]). We also set T(0) = {g € 
/C : { 0 , 5 } € T}. It is easy to see that is ortho-complemented if and only if T(0) is 
ortho-complemented. We have 

Theorem 7 ([28]). Let T be a closed symmetric relation in a Krein space 1C such that T 0Q 
is ortho-complemented. Then T, = T fl (TY)^ is an operator with DITf) = D(T) and 
R(T S ) C T(0) ± . We also have 

(i) a p (T) = o p {T s ), 

(ii) <j(T) = (7(T s ), and 

(Hi) T is a selfadjoint relation if and only if T s is a selfadjoint operator in T(0) _L . 

Hence, we can reduce S to an operator if the set 

5(0) = {[(y n )] : lim \B\ 1 ^ 2 A~ 1 By n € Ker B} 

n 


is ortho-complemented. 

If, for instance, R{B) n HKerH = {0}, or if Ker B is T-invariant, 5(0) is ortho- 
complemented . 

If do = dim Ker B < 00 and if 5(0) is ortho-complemented, we can also obtain Rayleigh- 
Ritz-like upper bound for the eigenvalues of 5(0) in terms of the non-zero eigenvalues of 
the matrix pair (Ay, By), where V C D(A), in the case when the dimension of V is greater 
then the sum of do an d the spectral shift of S s = 5 n (5(0)) _L . We omit the details. 

Acknowledgement: The author is very grateful to anonymous referees for extremely useful 
comments and suggestions. 
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